feat(amazonq): Add Pin Context Tests and Update to pinContextHelper #7829

surajrdy-aws · 2025-08-06T18:39:08Z

Problem

We have yet to implement the initial P0 Pin Context Tests. The pinContextHelper does not mesh well with sub menu items when clicking through different prompts to use.

Solution

I have implemented the File Context and Prompt Context Test. The Pin Context test with the @workspace was previously implemented. I have implemented the helpers: getSubMenuItems and clickSubMenuItem in the pinContextHelper. All tests still run together.

Test Suite:

View Context Options
Reusable Prompt (New)
Add File as context (New)
Add Workspace as Context

Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
Your code changes must meet the guidelines in CONTRIBUTING.md.
License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

github-actions · 2025-08-06T18:39:33Z

This pull request implements a feat or fix, so it must include a changelog entry (unless the fix is for an unreleased feature). Review the changelog guidelines.
- Note: beta or "experiment" features that have active users should announce fixes in the changelog.
- If this is not a feature or fix, use an appropriate type from the title guidelines. For example, telemetry-only changes should use the telemetry type.

laura-codess

LGTM

laura-codess · 2025-08-10T18:38:12Z

looks like we need to rebase the latest changes based on the github actions

Hweinstock · 2025-08-12T15:07:46Z

feature --> feat

Invalid pull request title: `feature(amazonq): Add Pin Context Tests and Update to pinContextHelper`

* Problem: invalid type "feature"
* Expected format: `type(scope): subject...`
    * type: one of (build, ci, config, deps, docs, feat, fix, perf, refactor, revert, style, telemetry, test, types)
    * scope: lowercase, <30 chars
    * subject: must be <100 chars
    * documentation: https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#pull-request-title
* Hint: *close and re-open the PR* to re-trigger CI (after fixing the PR title).

Hweinstock · 2025-08-12T19:35:11Z

packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts

+export async function getSubMenuItems(webview: WebviewView): Promise<{ items: WebElement[]; labels: string[] }> {
+    try {
+        const menuList = await waitForElement(webview, By.css('.mynah-detailed-list-items-block'))
+        await sleep(3000)


Why is this additional sleep needed?

Hweinstock · 2025-08-12T19:36:42Z

packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts

+                console.log('Menu item found:', labelText)
+            } catch (e) {
+                labels.push('')
+                console.log('Could not get text for menu item')


should we fail here? If we can't find an item in the menu isn't that indicative of a bug?

Hweinstock · 2025-08-12T19:36:57Z

packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts

+
+        return { items: menuListItems, labels }
+    } catch (e) {
+        console.error('Error getting Pin Context menu items:', e)


same question here.

Hweinstock · 2025-08-12T19:37:35Z

packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts

+export async function clickSubMenuItem(webview: WebviewView, itemName: string): Promise<boolean> {
+    try {
+        const menuList = await waitForElement(webview, By.css('.mynah-detailed-list-items-block'))
+        await sleep(3000)


Why is this sleep needed? Shouldn't it be covered by waitForElement?

Hweinstock · 2025-08-12T19:37:58Z

packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts

+                    return true
+                }
+            } catch (e) {
+                continue


same question as above about errors.

Hweinstock · 2025-08-12T19:38:42Z

packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts

+        console.log(`Pin Context menu item not found: ${itemName}`)
+        return false
+    } catch (e) {
+        console.error(`Error clicking Pin Context menu item ${itemName}:`, e)


If this fails I think we should throw instead of returning a boolean.

Hweinstock · 2025-08-12T19:39:12Z

packages/amazonq/test/e2e_new/amazonq/tests/pinContext.test.ts

+        await createPrompt.click()
+        await clickPinContextButton(webviewView)
+        await clickPinContextMenuItem(webviewView, 'Prompts')
+        await clickSubMenuItem(webviewView, 'test')


my understanding is that this returns a boolean false if it fails, but this won't cause the test to fail.

Hweinstock · 2025-08-12T19:39:23Z

packages/amazonq/test/e2e_new/amazonq/tests/pinContext.test.ts

        await clickPinContextMenuItem(webviewView, '@workspace')
    })
+    it('Prompts Context Test', async () => {


same idea before on test labels.

Hweinstock · 2025-08-13T17:45:46Z

Looks like there is some duplicate logic:

[
  {
    first: 'https://github.com/aws/aws-toolkit-vscode/blob/1a83a0885221d1ebb4bb4dc2951b94ab0d603616/packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts#L103-L120',
    second: 'https://github.com/aws/aws-toolkit-vscode/blob/1a83a0885221d1ebb4bb4dc2951b94ab0d603616/packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts#L50-L67',
    numberOfLines: 18
  },
  {
    first: 'https://github.com/aws/aws-toolkit-vscode/blob/1a83a0885221d1ebb4bb4dc2951b94ab0d603616/packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts#L127-L140',
    second: 'https://github.com/aws/aws-toolkit-vscode/blob/1a83a0885221d1ebb4bb4dc2951b94ab0d603616/packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts#L73-L85',
    numberOfLines: 14
  }
]

but appears to be some overlapping comments. Could be worth trying to reduce duplication here, but don't see it as a blocker.

Hweinstock · 2025-08-13T17:46:18Z

packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts

+    for (const button of buttons) {
+        const label = await button.findElement(By.css('.mynah-button-label'))
+        const labelText = await label.getText()
+        console.log('THE BUTTON TEXT LABEL IS:', labelText)


Was this for debugging? Do we want to this run on text execution.

Oh do you mean the console.log statements?

Hweinstock · 2025-08-13T17:46:28Z

packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts

-        const menuListItems = await menuList.findElements(By.css('.mynah-detailed-list-item.mynah-ui-clickable-item'))
-        const labels: string[] = []
+    const menuList = await waitForElement(webview, By.css('.mynah-detailed-list-items-block'))
+    // TODO: Fix the need for a sleep function to be required at all.


Hweinstock · 2025-08-13T17:46:50Z

packages/amazonq/test/e2e_new/amazonq/helpers/pinContextHelper.ts

+        const nameElement = await textWrapper.findElement(By.css('.mynah-detailed-list-item-name'))
+        const labelText = await nameElement.getText()
+        labels.push(labelText)
+        console.log('Menu item found:', labelText)


Out of curiosity, how noisy are the logs when running the full test suite?

The logs aren't too noisy, I actually think they're helpful to see the process of the test in my opinion

Pin Context Test and Update to pinContextHelper

3bf85a6

surajrdy-aws requested a review from a team as a code owner August 6, 2025 18:39

Merge branch 'feature/ui-e2e-tests' into pinContext

d9b13a0

laura-codess approved these changes Aug 10, 2025

View reviewed changes

surajrdy-aws changed the title ~~feature(amazonq): Add Pin Context Tests and Update to pinContextHelper~~ feat(amazonq): Add Pin Context Tests and Update to pinContextHelper Aug 12, 2025

surajrdy-aws closed this Aug 12, 2025

surajrdy-aws reopened this Aug 12, 2025

Merge branch 'feature/ui-e2e-tests' into pinContext

f48bfdf

Hweinstock reviewed Aug 12, 2025

View reviewed changes

surajrdy-aws added 2 commits August 13, 2025 13:24

fix: update code from PR comments

b190cde

Merge branch 'feature/ui-e2e-tests' into pinContext

4252646

Hweinstock reviewed Aug 13, 2025

View reviewed changes

feat(amazonq): Add Pin Context Tests and Update to pinContextHelper #7829

Are you sure you want to change the base?

feat(amazonq): Add Pin Context Tests and Update to pinContextHelper #7829

Conversation

surajrdy-aws commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Uh oh!

github-actions bot commented Aug 6, 2025

Uh oh!

laura-codess left a comment

Choose a reason for hiding this comment

Uh oh!

laura-codess commented Aug 10, 2025

Uh oh!

Hweinstock commented Aug 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hweinstock commented Aug 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

surajrdy-aws commented Aug 6, 2025 •

edited

Loading